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[ABSTRACT] 



[Abstract] 

The present invention provides an apparatus and method for retrieving face 
images using combined component descriptors, which generates lower-dimensional 
face descriptors by synthesizing component descriptors for facial components into a 
single face descriptor, thus enabling precise face image retrieval while reducing the 
amount of processed data and retrieval time. 

Additionally, in the apparatus and method of the present invention, the joint 
retrieval method that utilizes an input face image and training face images similar to 
the input face image as comparison references at the time of face retrieval, thus 
providing a relatively high face retrieval rate. 

[Representative Drawing] 
FIG. 1 

[Index Keywords] 
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[SPECIFICATION] 



[Title of the Invention] 

Method of Retrieving Facial Image Using Combined Facial Component 

Descriptor and Apparatus Thereof 

[Brief Description of the Drawings] 

The above and other aspects, features and advantages of the present invention 

will become apparent from the following description of exemplary embodiments given 

in conjunction with the accompanying drawings, in which: 

FIG 1 is a diagram showing the construction of apparatus for retrieving face 
images according to an embodiment of the present invention; 

FIG. 2 is a flowchart showing a method of retrieving face images according to 
an embodiment of the present invention; 

FIG 3 is a block diagram showing the face image retrieving method according 
to the embodiment of the present invention; 

FIG 4 is a flowchart showing a process of determining similarities according to 
an embodiment of the present invention; 

FIGS. 5A and 5B are a view showing a process of dividing a face image 
according to an embodiment of the present invention; and 

FIG 6 is a table of experimental results obtained by carrying out experiments 
using a conventional face retrieval method and the face retrieval method of the present 
invention. 

[Detailed Description of the Invention] 
[Object of the Invention] 

[Field of the Invention and Description of the Related Art] 

The present invention relates generally to an apparatus and method for 
retrieving face images, using combined component descriptors. 

Generally, in face image retrieval technologies, a face image input by a user 



(hereinafter referred to as "queried face image") is compared with face images stored in 
a face image database (DB) (hereinafter referred to as "trained face images") to thereby 
retrieve from the DB a trained face image identical with or the most similar to the 
queried face image as inputted. 

In order to obtain a retrieval result as accurate as possible when retrieving a 
stored face image the most similar to the queried face image, among stored face images, 
face images of each person must be databased by means of features that can represent 
the best identify of the person having the face images, irregardless of illumination, 
posture of or facial expression of the person. Considering that the database would be 
of a large volume, storing therein a large number of face images relative to a lot of 
persons, a method of determining the similarity in a simple manner is necessary. 

In general, a face image is comprised of pixels. These pixels are presented in 
one column vector and the dimensionality of the vector is considerably large. For this 
reason, various researches have been carried out, to represent face images using a small 
amount of data while maintaining precision and to find out the most similar face image 
with a small number of calculations when retrieving a stored face image the most 
similar to the queried face images from a face image DB. 

As those methods that can represent face images with a small amount of data 
and retrieve a face image with a small number of calculations while obtaining accurate 
retrieval results, there are currently PCA, LDA and the like. The PCA stands for 
"Principal Components Analysis," using an eigenface, and the LDA stands for "Linear 
Discriminant Analysis" wherein the projection W (transformation matrix) to maximize 
between-class (person) scatters and to minimize within-class scatter (between-various 
images of a person) is determined, and represent a face image with a predetermined 
descriptor by use of the determined projection W. 

Additionally, there is used a method of retrieving face images in such a way 
that an entire face image is divided into several facial components, e.g., eyes, a nose and 
a mouth, rather than being represented as it is, wherein feature vectors are extracted 



from the facial components and the extracted feature vectors are compared with each 
other with the weights of the components being taken into account. 

A method of retrieving face images by applying the LDA method to divided 
facial components is described in Korean Patent Appln. 10-2002-0023255 entitled 
"Component-based Linear Discriminant Analysis (LDA) Facial Descriptor." 

However, since those conventional methods compare all the feature vector data 
of respective components with one another, the amount of data that are compared with 
one another is considerably increased when training face images of high capacity are 
compared with one another, so the processing of data becomes inefficient and the 
processing time of data is lengthened. Additionally, those conventional methods do 
not sufficiently consider correlations between the facial components, and the precision 
of retrieval is insufficient. 

[Technical Aspects to be achieved by the Invention] 

Accordingly, the present invention has been made keeping in mind the above 
problems occurring in the prior art, and an object of the present invention is to provide 
an apparatus and method for retrieving face images using combined component 
descriptors, which generates lower-dimensional face descriptors by combining 
component descriptors generated with respect to facial components and compares the 
lower-dimensional face descriptors with each other, thus enabling precise face image 
retrieval while reducing the amount of data and retrieval time required for face image 
retrieval. 

Another object of the present invention is to provide an apparatus and method 
for retrieving face images using combined component descriptors, which utilizes an 
input query face image and training face images similar to the input query face image as 
comparison references at the time of face retrieval, thus providing a relatively high face 
retrieval rate. 

[Embodiments and Operations of the Invention] 

In order to accomplish the above object, the present invention provides an 



apparatus for retrieving face images using combined component descriptors, including 
an image division unit for dividing an input image into facial components, a LDA 
transformation unit for LDA transforming the divided facial components into 
component descriptors of the facial components, a vector synthesis unit for synthesizing 
the transformed component descriptors into a single vector, a Generalized Discriminant 
Analysis (GDA) transformation unit for GDA transforming the single vector into a 
single face descriptor, and a similarity determination unit for determining similarities 
between an input query face image and face images stored in an face image DB by 
comparing a face descriptor of the input query face image with face descriptors of the 
face images stored in the face image DB. 

According to another embodiment of the present invention, an apparatus for 
retrieving face images using combined component descriptors comprises an image 
division unit for dividing an input image into facial components, a first Linear 
Discriminant Analysis (LDA) transformation unit for LDA transforming the divided 
facial components into component descriptors of the facial components, a vector 
synthesis unit for synthesizing the transformed component descriptors into a single 
vector, a second LDA transformation unit for LDA transforming the single vector into a 
single face descriptor, and a similarity determination unit for determining similarities 
between an input query face image and face images stored in an face image database 
(DB) by comparing a face descriptor of the input query face image with face descriptors 
of the face images stored in the face image DB. 

In order to accomplish the above object, the present invention provides a 
method of retrieving face images using combined component descriptors, including the 
steps of dividing an input image into facial components, LDA transforming the divided 
facial components into component descriptors of the facial components, synthesizing 
the transformed component descriptors into a single vector, GDA transforming the 
single vector into a single face descriptor, and determining similarities between an input 
query face image and face images stored in an face image DB by comparing a face 



descriptor of the input query face image with face descriptors of the face images stored 
in the face image DB. The step of LDA transforming the divided facial components 
comprises the steps of LDA transforming the divided facial components into component 
descriptors of the facial components, and vector normalizing the transformed 
component descriptors into a one-dimensional vector, wherein the LDA transforming or 
the GDA transforming is carried out using a transformation matrix or a transformation 
coefficient calculated by training the face images stored in the image DB. 

In addition, the present invention provides a method of retrieving face images 
using combined component descriptors, including the steps of dividing an input image 
into facial components, LDA transforming the divided facial components into 
component descriptors of the facial components, synthesizing the transformed 
component descriptors into a single vector, LDA transforming the single vector into a 
single face descriptor, and determining similarities between an input query face image 
and face images stored in an face image DB by comparing a face descriptor of the input 
query face image with face descriptors of the face images stored in the face image DB. 

Reference now should be made to the drawings, in which the same reference 
numerals are used throughout the different drawings to designate the same or similar 
components. 

First, the LDA method applied to the present invention is described below. 
The LDA method is disclosed in the paper of T. K. Kim, et al., "Component-based 
LDA Face Descriptor for Image Retrieval", British Machine Vision Conference 
(BMVC), Cardiff, UK, Sep. 2-5, 2002. 

If a training method, such as the LDA method, is employed, the variations of 
illumination and poses can be eliminated during encoding. In particular, the LDA 
method can effectively process a face image recognition scenario in which two or 
more face images are registered, which is an example of identity training. 

Meanwhile, the LDA method is the method that can effectively represent 
between-class disperse (disperse between classes (persons)) having different identities 



and, therefore, can distinguish the variation of face images caused by the variations of 
identities from the variations of face images caused by the variations of other factors, 
such as the variations of illumination and impressions. LDA is a class specific method 
in that it represents data to be useful to classification. This method can be 
accomplished by calculating a transformation that that maximizes between-class 
scatter while minimizing within-class scatter. Accordingly, when a person tries to 
recognize a face image under an illumination condition different from that at the time 
of registration, the variation of a face image results from the variations of illumination, 
it can be determined that the varied face image belongs to the same person. Here is 
the brief mathematical description of LDA. Given a set of N images {xi, X2, ■ • xn} 
each belonging to one of class C {Xi, X 2 , • • X c }, LDA selects a linear 
transformation matrix W so that the ratio of the between-class scatter to the within- 
class scatter is maximized. 

The between-class scatter and the within-class scatter can be represented by 
the following equation 1 . 



/=1 



(1) 

where /u denotes the mean of entire images, ju t denotes the mean image of class 
X f , and N, denotes the number of images in class X t . If the within-class scatter 
matrix S w is not singular, LDA finds an orthonormal matrix W op t that maximizes the 
ratio of the determinant of the between-class scatter matrix to the determinant of the 
within-class scatter matrix. That is, the LDA projection matrix can be presented by 

W T S R W 



W opl =argmax 



= [w, w 2 ■■■ w m ] 



(2) 



The set of solution {w i \ i = 1, 2, m) is that of generalized eigenvectors of 
S B and S w corresponding to the m largest engenvalues {A, | i = 1, 2, m} . 
The LDA face descriptor is described below. 

Under the present invention, in order to take advantages of both a desirable 
linear property and robustness to image variation of the component-based approach, 
LDA is combined with the component-based representation. The LDA method is 
applied to divided facial components respectively, by which the precision of retrieval 
is improved. 

For a training data set, an LDA transformation matrix is extracted. Given a 
set of N training images {x l9 x 2 , • • x N }, all the images are divided into L facial 

components by a facial component division algorithm. All patches of each 
component are gathered and are represented in vector form: the k- th component is 

denoted as {z,* , z k , • z N k } . Then, for the set of each facial component, a LDA 

transformation matrix is trained. For the k- facial component, the corresponding 
LDA matrix W k is computed. Finally, the set of LDA transformation matrices 
{W x ,W 2 , • • -,W L } is stored to be used for a training stage or retrieval stage. 

For the training face images, L vectors {z 1 , z 2 , • • •, z L } corresponding to facial 
component patches are extracted from a face image jc. A set of LDA feature vectors 
y - y 2 y * ' "> y L ) * s extracted by transforming the component vectors by the 
corresponding LDA transformation matrices, respectively. The feature vectors are 
computed by y k =(W k ) T z k ,k = l,2, -- 9 L. 

Consequently, for the component-based LDA method, a face image x is 
compactly represented by a set of LDA feature vectors, that is, component descriptors 

In conclusion, in order to apply the LDA method, LDA transformation matrices 
W k must be computed for the facial components, and later input query face images 
are LDA converted by the calculated LDA transformation matrix W k using 



y k = (W k ) T z k . 

Hereinafter, the Generalized Discriminant Analysis (GDA) method applied to 
the present invention is described. The GDA method is disclosed in the paper of 
BAUDAT G, et al., "Generalized Discriminant Analysis Using a Kernel Approach", 
Neural Computation, 2000. 

GDA is a method designed for non-linear feature extraction. The object of 
GDA is to find a non-linear transformation that maximizes the ratio between the 
between-class variance and the total variance of transformed data. In the linear case, 
maximization of the ratio between the variances is achieved via the eigenvalue 
decomposition similar to LDA. 

The non-linear extension is performed by mapping the data from the original 
space Y to a new high dimensional feature space Z by a function 0: Y—>Z. The 
problem of high dimensionality of the new space Z is avoided using a kernel function k: 
YxY—>R. The value of the kernel function k(y i} yj) is equal to the dot product of non- 
linearly mapped vectors O (yO and O(yj), i.e., k(y it yj) = &(yi) T( l>(yj), which can be 
evaluated efficiently without explicit mapping the data into the high dimensional space. 

It is assumed that yuj denotes the /- th training pattern of k- ih class, M is the 

M 

number of classes, TV, is the number of patterns in the z- th class, and N = ^ N k denotes 

k=\ 

the number of all patterns. If it is assumed that the data are centered, the total scatter 
matrix of the non-linearly mapped data is 

1 M H k 

The between-class scatter matrix of non-linearly mapped data is defined as 

1 M 

S B =—"ZN K Q>(^ k )a>(ju k ) T , 



where 0(ju k ) = — £ ®(y k i ) . 

The aim of the GDA is to find such projection vectors w eZ which maximize the 

ratio 

(3) 

w S T w 

It is well known that the vectors weZ maximizing the ratio, such as Equation 3, 
can be found as the solution of the generalized eigenvalue problem 

XS r w - S B w (4) 
where X is the eigenvalue corresponding to the eigenvector w. 

To employ the kernel functions all computations must be carried out in terms of 
dot products. To this end, the projection vector w is expressed as a linear combination of 
training patterns, i.e., 

^zix^ov) (5) 

k=\ /=l 

where a^j are some real weights. Using Equation 5, Equation 3 can be expressed as 
. a T KWKa 

A — (o) 

a 7 KKa 

where the vector a^(ctk), k=l,..., M and a k =(a kJ ) ) i=l,...,N k , The kernel matrix K 
(NxN) is composed from the dot products of non-linearly mapped data, i.e., 

K=(Kk,i)k=i, ...M /=/ m (7) 

where K kJ = (k(y knyij ))^ ^ yssl N( . 

The matrix W (NxN) is a block diagonal matrix 

W^WOk^M (8) 



where k- tU matrix Wk on the diagonal has all elements which are equal to — . 

Solving the eigenvalue problem Equation 6 yields the coefficient vectors a that 
define the projection vectors w eZ. A projection of a testing vector y is computed as 

LI 

*=1 /=1 



As mentioned above, the training vectors are supposed to be centered in the 
feature space Z. The centered vector &(y) ' is computed as 

1 M N* 

o(^y=<i>(^--£|;o(^) do) 

which can be done implicitly using the centered kernel matrix K r (instead of K) since the 
data appears in terms of dot products only. The centered kernel matrix K' is computed 

as 

K'=K- — IK- — KI l —IKI (11) where 

N N N 

matrix / (NxN ) has all elements equal to /. Similarly, a testing vector y must be 
centered by Equation 10 before projecting by Equation 9. Application of Equations 10 
and 9 to the testing vector is equivalent to using the following term for projection 

^oOO'-ZEAXvw.jO + a (12) 

k=\ i=l 

The centered coefficients are computed as 



and bias b as 



b = - — JKJa + ~JceJKJ (14) 
N N 



where the column vector J (Nxl) has all terms equal to L 

In conclusion, to apply the GDA method, a kernel function to be used should 
previously be specified, transformation coefficients p and b should be computed, 
and a query face image input later is transformed through the use of Equation 12 using 
the computed transformation coefficients f5 and b . 

The present invention proposes to synthesize feature vectors for all facial 
components (i.e., component descriptors) calculated by LDA transformation 
(hereinafter referred to as a "first LDA transformation") into a single vector 

X = [y]yf •">'/'] an d to extract a related feature vector (i.e., a face descriptor fj) 

through LDA transformation or GDA transformation (hereinafter referred to as a 
"second LDA/GDA transformation"). The apparatus and method for retrieving face 
images using combined component descriptors preconditions training according to the 
following '1. Training Stage 5 , and '2. Retrieval Stage' is performed when a query face 
image is input. 

1 . Training Stage 

A. Training face images Xj are each divided into L face components according 
to an image division algorithm and are trained, and first LDA transformation matrices 
W k (k=l, 2, • • L) are calculated for the L facial components. 

B. The training face images Xj are first LDA transformed using the calculated 
W k (k=l, 2, • • L) and equation y k =(W k ) T z k , and LDA component descriptors 

y) > yf 9 ■ • m 9 y!' are calculated. 

C. With respect to each of the training face images Xj, the LDA component 
descriptors y) , y] , • • - 9 yf are vectors normalized and synthesized into a single 

vector^ =[y]y?~-y-l 



The vector normalization is performed using Equation a' = ^-r where a 

Ml 

denotes a vector with a length of n. 

D. A transformation matrix or transformation coefficient required for the 
second transformation (LDA or GDA) is calculated by training the single vectors. 

When the second LDA transformation is applied, a second LDA transformation 
matrix W for the single vectors is calculated. When the second GDA transformation 
is applied, a kernel function is specified and transformation coefficients P and b 
depending upon the kernel function specified by the training are calculated. 

E. With respect to the training face images Xj, face descriptors f\ to which the 
first LDA transformation and the second LDA/GDA transformation have been applied 
are calculated using the calculated transformation matrix or calculated transformation 
coefficients. 

2. Retrieval Stage 

A. An input query x is divided into L face components according to an image 
division algorithm. The L divided face components are first LDA transformed using 
first LDA transformation matrices W k (k=l, 2, • • L) calculated for the L facial 
components in the training stage. 

B. LDA component descriptors y] 9 yf 9 • • * 9 yf with respect to the input query 

face image x are vectors normalized and synthesized into^ = [y]yf -~yf ]. 

C. In the case where the second LDA transformation is applied, the single 
vector is second LDA transformed into a face descriptor f using the second LDA 
transformation matrix in the training stage. In the case where the second GDA 
transformation is applied, the single vector is second GDA transformed into a face 
descriptor f using a specified kernel function and training-specified transformation 
coefficients P and b. 



D. The similarities are determined between the face descriptor f calculated with 
respect to the input query face image x and the face descriptors f\ of the training face 
images calculated in 'E' of the training stage according to a certain similarity 
determination method. 

For reference, the transformation matrices, including the first LDA 
transformation matrices W k and the second LDA transformation matrices W 2nd 
calculated in the training stage, and the transformation coefficients P and b used for 
the second GDA transformation should be calculated before the retrieval stage, but the 
face descriptor fj (hereinafter z=f) may be calculated and stored in the training stage, 
or may be calculated together with an input query face image when the query face 
image is input. 

An entire procedure of the present invention is described in detail with 
reference to the accompanying drawings. 

FIG. 1 is a diagram showing the construction of apparatus for retrieving face 
images according to an embodiment of the present invention. 

The face image retrieving apparatus of the embodiment of the present 
invention may be divided into a cascaded LDA transformation unit 10, a similarity 
determination unit 30, and an image DB 30 in which training face images are stored. 
A face descriptor z of an input query face image is calculated through the cascaded 
LDA transformation unit 10. The similarity determination unit 20 determines the 
similarities between the calculated face descriptor z of the query face image and face 
descriptors z \ of the training face images stored in the image DB 30 according to a 
certain similarity determination method, and outputs retrieval results. The output 
retrieval results are a training face image with the highest similarity, or training face 
images that have been searched for and are arranged in the order of similarities. 

The face descriptors z \ are previously calculated in a training stage and 
stored in the image DB 30, or are calculated by inputting a training face image 
together with a query face image to the cascaded LDA transformation unit 10 when 



the query face image is input. 

A method of determining similarity according to an embodiment of the present 
invention will be described later in the detailed description of FIG 4, 

The construction of the cascaded LDA transformation unit 10 is described in 
detail with reference to FIG. 1. The cascaded LDA transformation unit 10 includes 
an image input unit 100 for receiving a face image as shown in FIG. 5A ? and an image 
division unit 200 for dividing the face image received through the image input unit 
100 into L facial components, such as eyes, a nose and a mouth. An exemplary face 
image divided by the image division unit 200 is illustrated in FIG 5B. In FIG. 5B, 
the face image is divided into five components on the basis of eyes, a nose and a 
mouth, and the divided five components are partially overlapped with each other. 
The reason why the divided components are partially overlapped with each other is to 
prevent the features of a face from being lost by the division of the face image. 

L facial components divided by the image division unit 200 are LDA 
transformed into the component descriptors of the facial components by the first LDA 
transformation unit 300. The first LDA transformation unit 300 includes L LDA 
transformation units 310 for LDA transforming L facial components divided by the 
image division unit 200 into the component descriptors of the facial components, and 
L vector normalization units 320 for vector normalizing the component descriptors 
transformed by the LDA transformation units 310. As described above, the vector 
normalization of component descriptors is performed using the following equation 

where a denotes a vector having a length of n. 

The L LDA transformation units 310 LDA transform the components of an 
input query face image using a first LDA transformation matrix W k (k=l, 2, • • •, L) for 
each of the components stored in a transformation matrix/transformation coefficient 
DB 600 according to the training results of the training face images within the image 



DB 30. For example, when the component, including the forehead of FIG. 5B, is 1, 
that is, k=l, this component including the forehead is LDA transformed using W 1 . 
When the component, including the right eye of FIG. 5B, is 2, that is, k=2, this 
component, including the forehead, is LDA transformed using W 2 . 

For reference, in this embodiment, the L LDA transformation units 310 and the 
L vector normalization units 320 may be replaced with a single LDA transformation 
unit 310 and a single vector normalization unit 320 that can process a plurality of 
facial components in parallel or in sequence, respectively. 

L component descriptors vector normalized in the L vector normalization units 
320 are synthesized into one vector in a vector synthesis unit 400. The synthesized 
vector is formed by synthesizing L divided components, so it has L times of the 
dimensions of single component vector. 

A single vector synthesized in the vector synthesis unit 400 is LDA or GDA 
transformed in the second LDA transformation unit or the second GDA transformation 
unit 500 (hereinafter referred to as the "second LDA/GDA transformation unit). 

The second LDA/GDA transformation unit 500 calculates the face descriptor z 
by performing second LDA transformation using a second LDA transformation matrix 
W 2nd stored in the transformation matrix/transformation coefficient DB 600 (in the 
case of the second LDA transformation unit), or by performing second GDA 
transformation using a previously specified kernel function and training-specified 
training transformation coefficients P and b stored in the transformation 
matrix/transformation coefficient DB 600 according to the training results of the 
training face images within the image DB 30 (in the case of the second GDA 
transformation unit). 

After the face descriptor z of the query face image is calculated in the cascaded 
LDA transformation unit 10, the similarity determination unit 20 determines the 
similarities between the face descriptors Zj of the training face images stored in the 
image DB 30 and the calculated face descriptor z of the query face image according to 



a certain similarity determination method, and outputs retrieval results. The 
similarity determination method used in the similarity determination unit 20 may be a 
conventional method of simply calculating similarities by calculating a normalized- 
correlation between the calculated face descriptor z of the query face image and the 
face descriptors Zj of the training face images stored in the image DB 30, or a joint 
retrieval method to be described later with reference to FIG. 4. For reference the 
conventional method of calculating similarities d(zl, z2) by calculating the normalized 
correlation is performed using the following equation 



For reference, in the face image retrieving apparatus according to the 
embodiment of the present invention, all the modules of the apparatus may be 
implemented by hardware, part of the modules may be implemented by software, or 
all the modules may be implemented by software. Accordingly, it does not depart 
from the scope and spirit of the invention to implement the apparatus of the present 
invention using hardware or software. Further, it is apparent from the above 
description that the apparatus of the present invention is implemented by software and 
modifications and changes due to the software implementation of the apparatus are 
possible without departing from the scope and spirit of the invention. 

A method of retrieving face images using combined component descriptors 
according to an embodiment of the present invention is described with reference to 
FIGs. 2 and 3. 

FIG. 2 is a flowchart showing the face image retrieving method according to 
the embodiment of the present invention. FIG. 3 is a block diagram showing the face 
image retrieving method according to the embodiment of the present invention. 

When a query face image x is input to the image input unit 100, the query face 
image x is divided into L facial components according to a specified component 
division algorithm in the image division unit 100 at step S10. In the L LDA 



transformation unit 310 of the first LDA transformation unit 300, the L components of 
the input query face image are first LDA transformed using the first LDA 
transformation matrix W k (k=l, 2, • • •, L) stored in the transformation 
matrix/transformation coefficient DB 600 according to the training results of the 
training face images within the image DB 30 at step S20. 

The component descriptors CD1, CD2, • • •, CDL are vector normalized LDA 
transformed in the L LDA transformation unit 310 are vector normalized by the L 
vector normalization units 320 at step S30 5 and, thereafter, are synthesized into a 
single vector having dimensions at step S40. 

The single vector into which the component descriptors are synthesized is 
thereafter second LDA/GDA transformed by the LDA/GDA transformation unit 500 at 
step S50. 

The face descriptor z is calculated by performing the second LDA 
transformation matrix W 2nd calculated in the training stage in the case of the second 
LDA transformation unit 500, or by performing the second GDA transformation using 
a specified kernel function and training-specified transformation coefficients P and b 
in the case of the second GDA transformation unit. 

Thereafter, with respect to the input query face image x, the similarity 
determination unit 20 determines the similarities between the face descriptor z 
calculated in the second LDA/GDA transformation unit 500 and the face descriptors zi 
of the training face images stored in the image DB 30 according to a certain similarity 
determination method at step S60, and outputs retrieval results at step S70. As 
described above, the output retrieval results are a training face image with the highest 
similarity or training face images that have been searched for and are arranged in the 
order of similarities. The face descriptors Zj are previously calculated in a training 
stage and stored in the image DB 30, or are calculated by inputting a training face 
image together with a query face image to the cascaded LDA transformation unit 10 
when the query face image is input. 



The similarity determination method according to an embodiment of the 
present invention is described with reference to FIG. 4. 

In the embodiment of the present invention, the joint retrieval method is used 
as the similarity determination method. The joint retrieval method is the method in 
which the similarity determination unit 20 extracts the first similar face images from 
the image DB 30 falling within a certain similarity range on the basis of the input 
query face image in the order of similarities, extracts the second similar face images 
from the image DB 30 falling within a certain similarity range on the basis of the first 
similar face images, and utilizing the first and second similar face images as a kind of 
weights when determining the similarities between an input query face image and the 
training face images of the image DB. 

Although the above-described embodiment determines similarities by 
extracting the second similar face images, the present invention can utilize a plurality 
of similar face images including the third similar face images, the fourth similar face 
images, etc. 

The joint retrieval method according to the present invention is expressed as 
the following equation 15. 

M ML 

(15) 

where S fJ denotes the similarity between images i and j, h lst and h 2nd denote the 

indexes of face images highly ranked in first and second similar face images, 
respectively, and Joint S in the equation 15 denotes the final similarity between a 

query face image q and a certain training face image m stored in the image DB 30. 

For reference, Sy may be calculated using the conventional cross-correlation 

and 



In equation 15, S qm denotes the similarities between a query face image q 
and the face images m of the image DB 30, S % « k denotes the similarities between 

the query face image q and the first similar face images, S ^ denotes the 

similarities between the first similar face images and the face images m of the image 
DB 30, S hlMk h2mt/ denotes the similarities between the first similar face images and the 

second similar face images, S h2ndf m denotes the similarities between the second similar 

face images and the face images m of the image DB 30, M denotes the number of the 
first similar face images, and L denotes the number of the second similar face images 
with respect to each of the second similar face images. 

With reference to FIG 4, the similarity determination method according to an 
embodiment of the present invention is described below. 

After the first similarity determination in which the similarities are determined 
between a query face image and the training face images of the image DB 30 at step 
S61, first similar face images are extracted from the image DB 30 in the order of 
similarities according to the first similarity determination results at step S62. 

Thereafter, there is performed second similarity determination in which 
similarities are determined between the extracted first similar face images and the 
training face images of the image DB 30 at step S63, second similar face images with 
respect to each of the first similar face images are extracted from the image DB 30 in 
the order of similarities according to the second similarity determination results at step 

S64. A final similarity is determined by calculating the similarities S between 
the query face image and the training face images of the image DB at step S65. 



FIG. 6 is a table of experimental results obtained by carrying out experiments 
using a conventional face retrieval method and the face retrieval method of the present 
invention. In this table it can be seen that the face retrieval method of the 
embodiment of the present invention exhibited improved performance compared with 
the conventional face retrieval method. 

In the left column of FIG 6, 'Holistic' denotes the case where LDA 
transformation is applied to an entire face image without the division of the face 
image. 'LDA-LDA' denotes the face retrieval method according to an embodiment 
of the present invention in which second LDA transformation is applied after first 
LDA transformation. 'LDA-GDA' denotes the face retrieval method according to 
another embodiment of the present invention in which second GDA transformation is 
applied after the first LDA transformation. In 'LDA-GDA', a radial basis function 
was used as a kernel function. 

In the uppermost row of FIG. 6, 'experiment 1' was carried out in such a way 
that five face images with respect to each of 160 persons, that is, a total of 800 face 
images, were trained and five face images with respect to each of 474 persons, that is, 
a total of 2375 face images, were used as query face images. 'Experiment 2' was 
carried out in such a way that five face images with respect to each of 337 persons, 
that is, a total of 1685 face images, were trained and five face images with respect to 
each of 298 persons, that is, a total of 1490 face images, were used as query face 
images. 'Experiment 3' was carried out in such a way that a total of 2285 face 
images were trained and a total of 2090 face images were used as query face images. 

In accordance with the experimental results shown in FIG. 6, the face image 
retrieval methods according to the embodiments of the present invention have 
improved Average Normalized Modified Recognition Rates (ANMRRs) and False 
Identification Rates (FIRs) compared with the conventional face retrieval method. 

Although the preferred embodiments of the present invention have been 
disclosed for illustrative purposes, those skilled in the art will appreciate that various 



modifications, additions and substitutions are possible, without departing from the 
scope and spirit of the invention as disclosed in the accompanying claims. 

[Effects of the Invention] 

As described above, the present invention provides an apparatus and method 

for retrieving face images using combined component descriptors, which generates 
lower-dimensional face descriptors by synthesizing component descriptors for facial 
components into a single face descriptor, thus enabling precise face image retrieval 
while reducing the amount of processed data and retrieval time. 

Additionally, in the apparatus and method of the present invention, the joint 
retrieval method that utilizes an input face image and training face images similar to 
the input face image as comparison references at the time of face retrieval, thus 
providing a relatively high face retrieval rate. 



[Claims] 

1. An apparatus for retrieving face images using combined component 
descriptors, comprising: 

an image division unit for dividing an input image into facial components; 

a Linear Discriminant Analysis (LDA) transformation unit for LDA 
transforming the divided facial components into component descriptors of the facial 
components; 

a vector synthesis unit for synthesizing the transformed component descriptors 
into a single vector; 

a Generalized Discriminant Analysis (GDA) transformation unit for GDA 
transforming the single vector into a single face descriptor; and 

a similarity determination unit for determining similarities between an input 
query face image and face images stored in an face image database (DB) by comparing 
a face descriptor of the input query face image with face descriptors of the face images 
stored in the face image DB. 

2. An apparatus for retrieving face images using combined component 
descriptors, comprising: 

an image division unit for dividing an input image into facial components; 

a first LDA transformation unit for LDA transforming the divided facial 
components into component descriptors of the facial components; 

a vector synthesis unit for synthesizing the transformed component descriptors 
into a single vector; 

a vector synthesis unit for synthesizing each of the component descriptors into a 
single vector; 

a second LDA transformation unit for LDA transforming the single vector into a 
single face descriptor; and 

a similarity determination unit for determining similarities between an input 
query face image and face images stored in an face image database (DB) by comparing 
a face descriptor of the input query face image with face descriptors of the face images 
stored in the face image DB. 



3. The apparatus as set forth in claims 1 and 2, wherein the LDA transformation 
unit comprises: 

LDA transformation units for LDA transforming the divided facial components 
into component descriptors of the facial components; and 

vector normalization units for vector normalizing the transformed component 
descriptors into a one-dimensional vector. 

4. The apparatus as set forth in claims 3, wherein the LDA transformation units 
and vector normalization units are each provided for the divided facial components. 

5. The apparatus as set forth in claims 1 and 2, further comprising a 
transformation matrix/transformation coefficient DB for storing a transformation matrix 
or transformation coefficients calculated by training the face images stored in the image 
DB, 

wherein the LDA transformation unit or the GDA transformation unit performs 
LDA transformation or GDA transformation using the stored transformation matrix or 
transformation coefficients. 

6. The apparatus as set forth in claims 1 and 2, wherein the image DB stores 
face descriptors of the face images; and 

the comparing of the input query face image with the face images of the image 
DB is performed by comparing the face descriptor of the input query face image with 
the face descriptors of the face images stored in the image DB. 

7. The apparatus as set forth in claims 1 and 2 5 wherein the divided face 
components are partially overlapped with each other. 

8. The apparatus as set forth in claims 1 and 2, wherein the face components 
into which the input face image is divided comprises eyes, a nose and a mouth. 

9. The apparatus as set forth in claims 1 and 2, wherein the similarity 



determination unit extracts first similar face images similar to the input query face 
image and second similar face images similar to the first face images from the image 
DB, and determines similarities between the input query face image and the face images 
of the image DB using the similarities between the input query face image and the 
second similar face images. 

10. The apparatus as set forth in claim 9, wherein the determination of the 
similarities between the input query face image and the face images of the image DB is 
performed using the following equation 

M ML 
Joint * =1 * =1 /=1 

where denotes similarities between the input query face image q and the face 

images m of the image DB, q > hU ' k denotes similarities between the query face image q 

and the first similar face images, hl5 ' k - m denotes similarities between the first similar 

face images and the face images m of the image DB, h Xst k y h 2nJ i d eno tes similarities 
between the first similar face images and the second similar face images, 

h 2nd i, m d eno tes similarities between the second similar face images and the face images 

m of the image DB, M denotes a number of the first similar face images, and L denotes 
a number of the second similar face images with respect to each of the second similar 
face images. 

1 1 . A method of retrieving face images using combined component descriptors, 
comprising the steps of: 

dividing an input image into facial components; 

LDA transforming the divided facial components into component descriptors of 
the facial components; 

synthesizing the transformed component descriptors into a single vector; 
GDA transforming the single vector into a single face descriptor; and 



determining similarities between an input query face image and face images 
stored in a face image DB by comparing a face descriptor of the input query face image 
with face descriptors of the face images stored in the face image DB. 

12. A method of retrieving face images using combined component descriptors, 
comprising the steps of: 

dividing an input image into facial components; 

LDA transforming the divided facial components into component descriptors of 
the facial components; 

synthesizing the transformed component descriptors into a single vector; 

LDA transforming the single vector into a single face descriptor; and 

determining similarities between an input query face image and face images 
stored in a face image DB by comparing a face descriptor of the input query face image 
with face descriptors of the face images stored in the face image DB. 

13. The method as set forth in claims 11 and 12, wherein the step of LDA 
transforming the divided facial components comprises the steps of: 

LDA transforming the divided facial components into component descriptors of 
the facial components; and 

vector normalizing the transformed component descriptors into a one- 
dimensional vector. 

14. The method as set forth in claims 11 and 12, wherein the LDA transforming 
or the GDA transforming is carried out using a transformation matrix or a 
transformation coefficient calculated by training the face images stored in the image DB. 

15. The method as set forth in claims 11 and 12, further comprising the step of 
outputting the face images of the image DB retrieved based on the determined 
similarities. 

16. The method as set forth in claims 11 and 12, wherein the comparing of the 
input query face image with the face images of the image DB is performed by 



comparing the face descriptor of the input query face image with the face descriptors of 
the face images stored in the image DB. 

17. The method as set forth in claims 11 and 12, wherein the divided face 
components are partially overlapped with each other. 

18. The method as set forth in claims 11 and 12, wherein the face components 
into which the input face image is divided comprises eyes, a nose and a mouth. 

19. The method as set forth in claims 11 and 12, wherein the step of 
determining similarities comprises the steps of: 

extracting first similar face images similar to the input query face image and 
second similar face images similar to the first face images from the image DB; and 

determining similarities between the input query face image and the face 
images of the image DB using the similarities between the input query face image and 
the second similar face images. 

20. The method as set forth in claim 19, wherein the step of extracting the first 
and second similar face images comprises: 

the first similarity determination step of determining similarities between the 
input query face image and the face images of the image DB; 

the first similar face image extraction step of extracting the first similar face 
images in an order of similarities according to results of the first similarity 
determination step; 

the second similarity determination step of determining similarities between the 
first similar face images and the face images of the image DB; and 

the second similar face image extraction step of extracting the second similar 
face images for each of the first similar face images in an order of similarities according 
to results of the second similarity determination step. 

21. The method as set forth in claim 19, wherein the determining of similarities 
between the input query face image and the face images of the image DB is performed 



using the followingjjquatipi^ ^ ~ ^ c V <? <r 

*qjn -*q/n + Za^q^'k '^h u 'k,m + q,h lst k Zj^ h Ui k,h 2nd l '^h 2 " d I t m 

Joint *-i *-i '=1 

where ^ denotes similarities between the input query face image q and the 

face images m of the image DB, ***** denotes similarities between the query face 

image q and the first similar face images, hU ' k > m denotes similarities between the first 

S 

similar face images and the face images m of the image DB, ^"M 2 ""/ denotes 

similarities between the first similar face images and the second similar face images, 
S 

h 2n ''i,m d eno tes similarities between the second similar face images and the face images 
m of the image DB, M denotes a number of the first similar face images, and L denotes 
a number of the second similar face images with respect to each of the second similar 
face images. 
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